Web Survey Bibliography
Measurement effects are a major problem in mixed-mode surveys suggesting that the same respondent potentially provides different answers under different modes. Mixed-mode researchers therefore often need to know the average size of measurement effects (AME) for the questions of their interest. The present paper discusses estimation of AME using two different data collection approaches: a between subject and a within subject (repeated measures) design. Real-world data from an experiment with N=8,800 subjects in The Netherlands are presented. In the ‘between design’, subjects were randomly allocated to one mode only (Face-to-Face, Telephone, Mail, or Web). In the ‘within design’ subjects were first allocated as in the ‘between design’ and subsequently re-approached after some weeks in a reference mode (Face-to-Face) repeating a large number of questions. Unit nonresponse in both designs represents a threat to full randomization and thus to unbiased estimation of the AME, if confounders relate to the selection mechanism into mode conditions and the outcome variable. Statistical adjustment of missing data is a possible solution to this problem, but it is based on assumptions. Adjustment in ‘between designs’ assumes that the selection mechanism is ignorable given auxiliary variables. This is often contestable in practice, because some important confounders might not be observed. An advantage of ‘within designs’ is that it is more plausible to ignore the selection mechanism when conditioning on the repeated measurements. Thereby it is not problematic whether time-related changes of outcomes between measurement occasions occur, because these can be controlled using subjects who are allocated to the reference mode on both occasions (i.e., Face-to-Face). However, ‘within designs’ need to assume that measurements can be taken independently across time. We compare AME estimates from both designs for questions from the Dutch Crime Victimization Survey applying regression adjustment with propensity score strata as covariates or propensity score weighting.
Conference Homepage (abstract)
Web survey bibliography - Hox, J. (23)
- Mixed Mode Research: Issues in Design and Analysis; 2016; Hox, J.; De Leeuw, E. D.; Klausch, L. T.
- Internet Panels, Professional Respondents, and Data Quality; 2015; Matthijsse, S.; De Leeuw, E. D.; Hox, J.
- Selection error in single- and mixed mode surveys of the Dutch general population; 2015; Hox, J., Klausch, L. T., Schouten, B.
- Evaluating mixed-mode redesign strategies against benchmark surveys: the case of the Crime Victimization...; 2014; Klausch, L. T., Hox, J., Schouten, B.
- The use of within-subject experiments for estimating measurement effects in mixed-mode surveys ; 2014; Klausch, L. T., Schouten, B., Hox, J.
- Pret met panels [Fun online]; 2013; Roberts, A., de Leeuw, E. D., Hox, J., Klausch, L. T., de Jongh, A.
- Leuker kunnen wij het wel maken. Online vragenlijst design: standaard matrix of scrollmatrix (We can...; 2013; Roberts, A., de Leeuw, E. D., Hox, J., Klausch, L. T., de Jongh, A.
- Internet Coverage and Coverage Bias in Europe: Developments Across Countries and Over Time; 2013; Mohorko, A., de Leeuw, E. D.,Hox, J.
- Measurement Effects of Survey Mode on the Equivalence of Attitudinal Rating Scale Questions; 2013; Klausch, L. T., Hox, J., Hox, J., Schouten, B.
- Random versus Systematic Error in a Mixed Mode Online-Telephone Survey; 2013; Hox, J., Scherpenzeel, A., Boeve, A., Boeve, A., de Leeuw, E. D.
- Estimating Measurement Effects of Survey Modes From Between and Within Subject Designs; 2013; Klausch, L. T., Hox, J., Schouten, B.
- Does one really know?: Avoiding noninformative answers in a reliable way.; 2013; de Leeuw, E. D., Boevee, A., Hox, J.
- Design of Web Questionnaires: Matrix Questions or Single Question Formats ; 2012; de Leeuw, E. D., Hox, J., Klausch, L. T., Roberts, A., de Jongh, A.
- Question or Mode Effects in Mixed-Mode surveys: A Cross-cultural study in the Netherlands, Germany,...; 2012; de Leeuw, E. D., Nicolaas, G., Campanelli, P., Hox, J.
- Assessing Measurement Equivalence and Bias of Questions in Mixed-mode Surveys Under Controlled Sample...; 2012; Klausch, L. T., Hox, J., Schouten, B.
- The Representativity of Web Surveys of the General Population compared to Traditional Modes and Mixed...; 2012; Klausch, L. T., Schouten, B., Hox, J.
- Matrix vs. Single Question Formats in Web Surveys: Results from a large scale experiment; 2012; Klausch, L. T., de Leeuw, E. D., Hox, J., de Jongh, A., Roberts , A.
- Flexibility of Web Surveys: Probing 'do-not-know' over the Phone and on the Web; 2011; Hox, J., de Leeuw, E. D.
- Mode Effect or Question Wording? Measurement Error in Mixed Mode Surveys; 2011; de Leeuw, E. D., Hox, J., Scherpenzeel, A.
- Measurement Error in Mixed Mode Surveys: Mode or Question Format?; 2011; de Leeuw, E. D., Hox, J.
- Missing data; 2008; de Leeuw, E. D., Hox, J.
- The influence of advance letters on response in telephone surveys; 2007; de Leeuw, E. D., Callegaro, M., Hox, J., Korendijk, E., Lensvelt-Mulders, G. J.
- The effect of computer-assisted interviewing on data quality: A review.; 1995; de Leeuw, E. D., Hox, J., Snijkers, G.